Foot Depth Map Point Cloud Completion using Deep Learning with Residual Blocks

Fit is extremely important in footwear as fit largely determines performanceand comfort. Current footwear fit estimation mainly usesonly shoe size, which is extremely limited in characterizing theshape of a foot or the shape of a shoe. 3D scanning presents asolution to this, where a foot shape can be captured and virtuallyfit with shoe models. Traditional 3D scanning techniques have theirown complications however, stemming from their need to collectviews covering all aspects of an object. In this work we explore adeep learning technique to compete a foot scan point cloud frominformation contained in a single depth map view. We examine thebenefits of implementing residual blocks in architectures for this application,and find that they can improve accuracies while reducingmodel size and training time

Deep Learning 3D Scans for Footwear Fit Estimation from a Single Depth Map

In clothing and particularly in footwear, the variance in the size and shape of people and of clothing poses a problem of how to match items of clothing to a person. This is specifically important in footwear, as fit is highly dependent on foot shape, which is not fully captured by shoe size. 3D scanning can be used to determine detailed personalized shape information, which can then be used to match against product shape for a more per- sonalized footwear matching experience. In current implementations however, this process is typically expensive and cumbersome. Typical scanning techniques require that a camera capture an object from many views in order to reconstruct shape. This usually requires either many cameras or a moving camera system, both of which being complex engineering tasks to construct. Ideally, in order to reduce the cost and complexity of scanning systems as much as possible, only a single image from a single camera would be needed. With recent techniques, semantics such as knowing the kind of object in view can be leveraged to determine the full 3D shape given incomplete information. Deep learning methods have been shown to be able to reconstruct 3D shape from limited inputs in highly symmetrical objects such as furniture and vehicles. We apply a deep learning approach to the domain of foot scanning, and present meth- ods to reconstruct a 3D point cloud from a single input depth map. Anthropomorphic body parts can be challenging due to their irregular shapes, difficulty for parameterizing and limited symmetries. We present two methods leveraging deep learning models to pro- duce complete foot scans from a single input depth map. We utilize 3D data from MPII Human Shape based on the CAESAR database, and train deep neural networks to learn anthropomorphic shape representations. Our first method attempts to complete the point cloud supplied by the input depth map by simply synthesizing the remaining information. We show that this method is capable of synthesizing the remainder of a point cloud with accuracies of 2.92±0.72 mm, and can be improved to accuracies of 2.55±0.75 mm when using an updated network architecture. Our second method fully synthesizes a complete point cloud foot scan from multiple virtual view points. We show that this method can produce foot scans with accuracies of 1.55±0.41 mm from a single input depth map. We performed additional experiments on real world foot scans captured using Kinect Fusion. We find that despite being trained only on a low resolution representation of foot shape, our models are able to recognize and synthesize reasonable complete point cloud scans. Our results suggest that our methods can be extended to work in the real world, with additional domain specific data

Spatial Detection of Vehicles in Images using Convolutional Neural Networks and Stereo Matching

Convolutional Neural Networks combined with a state of the artstereo-matching method are used to find and estimate the 3D positionof vehicles in pairs of stereo images. Pixel positions of vehiclesare first estimated separately in pairs of stereo images usinga Convolutional Neural Network for regression. These coordinatesare then combined with a state-of-art stereo-matching method todetermine the depth, and thus the 3D location, of the vehicles. Weshow in this paper that cars can be detected with a combined accuracyof approximately 90% with a tolerated radius error of 5%,and a Mean Absolute Error of 5.25m on depth estimation for carsup to 50m away

IR Shape From Shading Enhanced RGBD for 3D Scanning

RGBD Cameras such as the Microsoft Kinect that can quickly provideusable depth maps have become very affordable, and thusvery popular and abundant in recent years. Beyond gaming, RGBDcameras can have numerous applications, including their use in affordable3D scanners. These cameras however are limited in theirability to capture finer details. We explore the use of additional3D reconstruction algorithms to enhance the depth maps producedfrom RGBD cameras, allowing them to capture more detail

Scaled Monocular Visual SLAM

The fundamental shortcoming underlying monocular-based localizationand mapping solutions (SfM, Visual SLAM) is the fact thatthe obtained maps and motion are solved up to an unknown scale.Yet, the literature provides interesting solutions to scale estimationusing cues from focus or defocus of a camera. In this paper, wetake advantage of the scale offered by image focus to properly initializeVisual SLAM with a correct metric scale. We provide experimentsshowing the success of the proposed method and discussits limitations

Automated Screening for Diabetic Retinopathy Using Compact Deep Networks

Diabetes is a chronic condition affecting millions of people worldwide.One of its major complications is diabetic retinopathy (DR),which is the most common cause of legal blindness in the developedworld. Early screening and treatment of DR prevents visiondeterioration, however the recommendation of yearly screening isoften not being met. Mobile screening centres can increasing DRscreening, however they are time and resource intensive becausea clinician is required to process the images. This process can beimproved through computer aided diagnosis, such as by integratingautomated screening on smartphones. Here we explore the useof a SqueezeNet-based deep network trained on a fundus imagedataset composed of over 88,000 retinal images for the purpose ofcomputer aided screening for diabetic retinopathy. The results ofthis neural network validated the viability of conducting automatedmobile screening of diabetic retinopathy, such as on a smartphoneplatform